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EXECUTIVE  SUMMARY 


This  report  presents  the  results  of  the  National  Airspace  System 
(NAS)  Operational  Test  and  Evaluation  (OT&E)  of  the  Terminal 
Doppler  Weather  Radar  (TDWR).  The  TDWR  is  a  C-band  pencil  beam 
doppler  weather  radar  with  a  narrow  beam  width  (0.5*),  high 
sensitivity,  optimized  for  detection  of  hazardous  weather  in  the 
airport  terminal  area.  It  provides  the  capability  for  the 
detection,  processing,  and  communication  of  hazardous  weather 
information  to  air  traffic  controllers  and  pilots. 

In  addition  to  the  OT&E  activities  of  ACT-320,  the  TDWR  underwent 
a  series  of  Shakedown  tests  performed  by  AOS-250  and  Remote 
Maintenance  Monitoring  System  (RMMS)  tests  by  ACT-330.  Those 
activities  are  covered  in  separate  reports . 

The  ACT-320  OT&E  activities  took  place  incrementally  at  a  number 
of  different  locations.  Appendix  A,  table  A-1,  TDWR  Test 
Activities,  outlines  the  time  and  place  of  each  of  these  test 
increments,  referred  to  as  Test  Activities,  TAl  through  TAIO. 

Initial  OT&E  of  the  TDWR  system  took  place  at  Oklahoma  City 
during  August  through  October  1992  (TAl).  Due  to  deficiencies 
found  in  the  system,  the  OT&E  testing  was  not  successfully 
completed.  In  addition  air  traffic  control  (ATC)  evaluation  had 
to  be  postponed  due  to  a  lack  of  appropriate  convective  weather 
and  because  the  unresolved  system  problems  would  render  the  TDWR 
unreliable  for  ATC  use. 

A  follow-up  Weather  Performance  Test  was  conducted  at  Oklahoma 
City  during  April  through  May  1993  (TA2)  to  evaluate  various 
Weather  Performance  Improvements  as  well  as  other  system 
corrections.  Even  though  many  Weather  Performance  and  system 
improvements  were  noted,  several  important  issues  remained 
unresolved. 

An  OT&E  Operational  Re-Test  was  conducted  at  Houston  in  April 
through  May  1993  (TA3).  This  was  basically  a  regression  test  to 
determine  progress  in  resolving  problems  uncovered  in  TAl  and 
TA2.  Unfortunately,  several  key  issues,  particularly  those 
involving  system  reliability,  remained  unresolved.  During  this 
same  period,  a  software  enhancement  known  as  "Build  4+"  was  also 
evaluated.  This  patch  enabled  Low-Level  Wind  Shear  Alert  System 
(LLWAS  II)  Center  Field  Wind  (CFW)  data  to  be  displayed  on  the 
TDWR  Ribbon  Display  Terminal  (RDT) . 

The  Reliability  Re-Test  and  ATC  evaluation  was  conducted  in 
Memphis  during  August  through  September  1993  (TA4).  In  spite  of 
the  fact  that  the  Reliability  Issue  still  remained  open,  the  ATC 
evaluation  was  successfully  completed. 

A  Reliability  Re-Test  was  again  conducted  in  Memphis  during 
February  through  March  1994  (TA5).  Although  a  significant 
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improvement  was  noted,  problems  with  the  Harris  computer  caused 
an  unsatisfactory  Mean  Time  Between  Failures  (MTBF)  and  Mean  Time 
Between  Critical  Failures  (MTBCF) . 

An  OT&E  of  the  Build  5A  Upgrade  (TDWR/LLWAS  II  Integration)  was 
conducted  at  Memphis  during  April  1994  (TA6).  This  test  indicated 
that  the  LLWAS  II  Integration  was  successfully  accomplished  with 
no  degradation  to  the  TDWR  system.  There  was  however,  a  sensor 
mapping  problem  which  is  explained  in  detail  in  appendix  B  and 
Service  Report  (SR)  IAH94-001.  An  ATC  evaluation  was  conducted  to 
assess  the  operational  impact  of  the  Build  5A  Upgrade. 

A  Checkout  of  the  Build  5B  Upgrade  (TDWR/LLWAS  III  Integration) 
was  conducted  at  Denver  in  January  1995  (TA7).  This  test  was 
designated  a  Checkout  rather  than  an  OT&E  because  of  the  unique 
configuration  at  Denver.  It  did  however,  provide  an  insight  into 
the  system  performance  for  a  large  scale  LLWAS  III  site. 

An  OT&E  of  the  Build  5B  Upgrade  for  an  LLWAS  II  site  was 
conducted  at  Kansas  City  in  January  1995  (TA8).  This  test 
indicated  that  the  Build  5B  Upgrade  was  accomplished  with  no 
degradation  to  the  TDWR  System,  however,  the  sensor  mapping 
problem  remained. 

ACT- 320  was  requested  to  conduct  a  regression  test  at  Denver  to 
assess  the  effect  of  a  software  fix  designed  to  minimize  the 
impact  of  a  anomaly  discovered  in  the  January  95  test.  This  test 
was  conducted  in  April  1995  (TA9). 

An  OT&E  of  the  Build  5B  Upgrade  for  an  LLWAS  III  site  was 
conducted  at  Orlando  in  May  1995  (TAIO).  This  test,  in 
conjunction  with  the  LLWAS  II  site  test  in  Kansas  City  (TA8) 
constituted  the  full  OT&E  of  the  TDWR  Build  5B  baseline. 

Whenever  anomalies  were  encountered  during  OT&E  testing,  SRs  were 
written  describing  the  event  and  the  circumstances  surrounding 
it.  (See  tables  A-4  and  A-5.)  Presently,  there  are  35  ACT-320 
generated  SRs  that  remain  open.  ACT-320  recommends  that  AND-420 
develop  a  plan  for  the  resolution  of  these  SRs. 

Although  the  sequence  of  these  tests  was  not  as  originally 
planned,  ACT-320  demonstrated  the  necessary  flexibility  to  adjust 
according  to  system  availability  and  mission  requirements. 

Problem  resolution  has  been  a  high  priority  for  ACT-320,  AND-420, 
and  AOS-250  and  this  is  reflected  in  the  present  product.  The 
Build  5B  Baseline  represents  a  dramatic  improvement  in  terms  of 
stability  and  reliability  over  the  product  tested  in  Oklahoma 
City  (TAl).  Those  problems  that  remain  will  be  discussed  in  the 
body  of  this  report.  Due  to  weather  and  resource  considerations, 
the  final  Weather  Performance  testing  has  not  yet  been 
accomplished.  A  supplement  to  this  report  will  be  provided  when 
Weather  Performance  results  are  available. 
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1.  INTRODUCTION. 

1 . 1  PURPOSE . 

The  purpose  of  this  report  is  to  document  test  results  from 
Terminal  Doppler  Weather  Radar  (TDWR)  Operational  Test  and 
Evaluation  (OT&E)  Integration  and  OT&E  Operational  tests, 
conducted  from  1992  to  present,  including  TDWR  Baseline 
Operational  Test  and  Evaluation  (OT&E)  testing  and  TDWR  Build  5A 
and  5B  OT&E  testing. 

This  report  also  presents  conclusions  and  makes  recommendations 
for  this  project. 

1 . 2  SCOPE . 

This  report  provides  background  information  on  the  TDWR  system, 
and  contains  a  comprehensive  description  of  all  OT&E  activities. 
Including  test  objectives,  participants,  locations,  dates,  data 
collection  and  analysis  methods,  and  test  results. 

This  report  describes  an  incremental  series  of  tests  that  were 
conducted  between  August  1992  and  May  1995  on  the  TDWR.  A 
summary  of  the  Test  Activities  which  took  place  incrementally  at 
a  number  of  different  locations,  is  provided  in  appendix  A,  table 
A-1.  A  list  of  participating  organizations  along  with  their 
functional  contributions  is  provided  in  appendix  A,  table  A-2. 
Data  collection  and  analysis  methods  as  well  as  test  results  are 
provided  in  appendix  B. 

This  report  also  contains  an  update  of  open  TDWR  Service  Reports 
(SR) .  (See  table  A-5) . 


2 .  REFERENCE  DOCUMENTS . 

2.1  Federal  Aviation  Administration  DOCUMENTS. 


2.1.1  Federal  Aviation  Administration 


Specifications . 


FAA-E-2806/1  Terminal  Doppler  Weather  Radar  Specification, 
November  12,  1992,  w/(SCN  1,  2/1/93;  SCN  2, 
4/9/93;  SCN  3,  7/22/93;  SCN  4,  7/25/94;  SCN  5, 
9/27/94;  SCN  6,  2/27/95;  SCN  7,  6/21/95). 


NAS-SS-1000  National  Airspace  System  (NAS)  System 

Specification,  Volume  I,  Functional  and 
Performance  Requirements  for  the  National  Airspace 
System  General,  October  1992. 


NAS-SS-1000 


NAS  System  Specification,  Volume  III,  Functional 
and  Performance  Requirements  for  the  Ground-to-Air 
Element,  February  1993. 


NAS-SS-1000 


NAS  System  Specification,  Volume  V,  Functional  and 
Performance  Requirements  for  the  National  Airspace 
System  Maintenance  and  Operations  Support  Element, 
October  1992. 

2.1.2  FAA  Standards ♦ 

FAA-STD-024B  Content  and  Format  Requirements  for  the 

Preparation  of  Test  and  Evaluation  Documentation, 
August  22,  1994. 

CT  1710.2B  Preparation  and  Issuance  of  Formal  Reports, 

Technical  Notes  and  Other  Documentation,  February 
13,  1990. 


2.1.3  Other  FAA  Publications. 


FAA  ORDER 
1810. IF 

FAA  ORDER 
1810. 4B 

NAS-IR- 
31023105 
Part  1 
Revision  C 

NAS-IR- 
31023105, 
Part  2 
Revision  A 

NAS-MD-110 


NAS-MD-790 


NAS-MD-793 

TDWR  MTP 


FAA  Acquisition  Policy,  March  19,  1993. 


FAA  NAS  Test  and  Evaluation  Policy,  October  22, 
1992. 

LLWAS  Phase  III  to  TDWR  Interface  Requirements, 
December  7,  1993. 


LLWAS,  Phase  II  to  TDWR  Interface  Requirements 
December  2,  1992. 


Test  and  Evaluation  (T&E)  Terms  and  Definitions 
for  the  National  Airspace  System,  March  27,  1987. 

Interface  Control  Document  (ICD),  Maintenance 
Processor  Subsystem  (MPS)  to  Remote  Monitoring 
Subsystems  (RMSs)  and  Remote  Monitoring  Subsystem 
Concentrators  (RMSCs). 

Remote  Maintenance  Monitoring  System  (RMMS) 
Functional  Requirements  for  the  RMS. 

Terminal  Doppler  Weather  Radar  (TDWR)  Master  Test 
Plan  (MTP),  February  12,  1990. 
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TECHNICAL 

NOTE 

CT-TN92/6 

TECHNICAL 

NOTE 

CT-TN94/2 

TECHNICAL 

NOTE 

CT-TN94/19 

TECHNICAL 

NOTE 

CT-TN94/59 

TEST 

PROCEDURES 

TEST 

PROCEDURES 

QUICK  LOOK 
REPORT 

QUICK  LOOK 
REPORT 


QUICK  LOOK 
REPORT 

QUICK  LOOK 
REPORT 

QUICK  LOOK 
REPORT 

QUICK  LOOK 
REPORT 


QUICK  LOOK 
MEMORANDUM 

QUICK  LOOK 
REPORT 


TDWR  Operational  Test  and  Evaluation  (OT&E) 
Integration  Test  Plan,  November  1992. 


TDWR  Build  5  Test  and  Evaluation  Master  Plan 
(TEMP),  May  1994. 


TDWR  Build  5A  Operational  Test  and  Evaluation 
(OT&E)  Integration  and  OT&E  Operational  Test 
Plan,  July  1994. 

TDWR  Build  5B  Operational  Test  and  Evaluation 
(OT&E)  Integration  and  OT&E  Operational  Test 
Plan,  March  1995. 

TDWR  Build  5A  OT&E  Integration  and  OT&E 
Operational  Test  Procedures,  April  1994. 

TDWR  Build  5B  OT&E  Integration  and  OT&E 
Operational  Test  Procedures,  November  1994. 

TDWR  OT&E  Integration  and  OT&E  Operational  Quick 
Look  Report,  November  12,  1992. 

TDWR  OT&E  Integration  and  OT&E  Operational 
Tests/Retests  Quick  Look  Report,  September  21, 
1993. 

TDWR  OT&E  Operational  Retest  Quick  Look  Report, 
April  8,  1994. 

TDWR  Build  5A  OT&E  Integration  and  OT&E 
Operational  Test  Quick  Look  Report,  June  1,  1994. 

TDWR  Denver  Configuration  Checkout  Test  Quick  Look 
Report,  January  23,  1995. 

TDWR  Build  5B  (LLWAS  II)  OT&E  Integration  and  OT&E 
Operational  Test  Quick  Look  Report,  February  10, 
1995. 

TDWR  Denver  Configuration  Checkout  Retest  Quick 
Look  Memorandum,  May  3,  1995. 

TDWR  Build  5B  (LLWAS  III)  OT&E  Integration  and 
OT&E  Operational  Test  Quick  Look  Report,  June  8, 
1995. 
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3.  SYSTEM  DESCRIPTION. 


3.1  MISSION  REVIEW. 

The  TDWR  is  one  project  of  the  NAS  Plan,  whose  overall  goal  is 
the  modernization  and  improvement  of  the  Government  systems 
supporting  aviation  commerce  in  the  United  States .  In  the  end- 
state  of  the  NAS  Plan,  the  TDWR  will  send  weather  product 
information  to  the  air  traffic  control  (ATC)  computers  at  the 
Tower  Control  Computer  Complex  (TCCC).  Also  in  the  end-state,  a 
mechanism  will  be  provided  to  transmit  TDWR  hazardous  weather 
information  directly  to  the  pilots.  The  end-users  of  TDWR 
outputs  are  local,  approach,  and  departure  controllers,  their 
supervisors,  and  pilots.  In  the  interim  NAS,  the  TDWR  product 
information  will  be  displayed  to  air  traffic  specialists;  i.e., 
controllers  and  controllers'  supervisors. 

The  primary  mission  of  the  TDWR  is  to  enhance  the  safety  of  air 
travel  through  the  timely  detection  and  reporting  of  hazardous 
wind  shear  in  and  near  the  terminal  approach  and  departure  zones 
of  an  airport.  Specific  sources  of  the  hazardous  wind  shear 
which  are  to  be  detected  are  microbursts  and  gust  fronts.  A 
secondary  mission  of  the  TDWR  is  to  improve  the  management  of  air 
traffic  in  the  terminal  area  through  the  forecast  of  gust  front 
Induced  wind  shifts  at  the  airport. 

The  TDWR  is  being  deployed  at  unmanned  locations,  visited  only 
for  preventive  and  corrective  maintenance.  The  TDWR  will  operate 
24  hours  a  day,  7  days  a  week,  except  when  shutdown  for 
maintenance.  Operator  interaction  is  from  the  Federal  Aviation 
Administration  (FAA)  Remote  Maintenance  Monitoring  System  (RMMS) 
via  the  TDWR  Remote  Monitoring  System  (RMS) .  Local  control  is 
through  the  Maintenance  Data  Terminal  (MDT) ,  also  via  the  RMS. 

The  TDWR  system  is  composed  of  four  functional  areas;  Radar  Data 
Acquisition  (RDA),  Radar  Product  Generation  (RPG),  Remote 
Monitoring  Subsystem  (RMS),  and  Display  Functional  Unit  (DFU) . 

The  RDA  performs  radar  data  collection,  weather  detection,  signal 
processing,  clutter  suppression,  control,  monitoring  as  well  as 
error  detection  and  handling.  The  RPG  performs  weather  product 
generation,  RDA  scan  control  and  external  user  output  generation. 
The  RMS  performs  system  performance  status,  monitoring, 
reporting,  maintenance  alert  and  alarm  processing,  and  fault 
isolation.  The  DFU  provides  display  and  control  of  weather 
products.  It  also  performs  the  archiving  function.  Archived  data 
will  consist  of  TDWR/Low-Level  Wind  Shear  Alert  System  (LLWAS)  II 
and/or  TDWR/LLWAS  III,  Geographic  Situation  Display  (GSD) /Ribbon 
Display  Terminal  (RDT)  blanking,  and  programmable  alarm  timeouts. 

The  RMS  function  implements  those  RMMS  features  necessary  to 
allow  remote  monitoring  of  the  TDWR  system  operation.  However, 
the  RMMS  system  is  presently  not  fully  operational  despite  the 
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fact  the  system  is  being  fielded.  For  this  reason,  the  TDWR 
system  also  incorporates  maintenance  features  which  allow  the 
system  to  be  efficiently  maintained  during  the  interim  period 
until  the  RMMS  becomes  operational.  The  system  is  also  designed 
in  a  modular  fashion  which  allows  the  system  to  be  repaired 
rapidly  when  malfunctions  occur  by  replacing  Line  Replaceable 
Units  (LRUs). 

3.2  TEST  SYSTEM  CONFIGURATION. 

3.2.1  BUILD  4. 

The  Build  4  OT&E  hardware  test  configuration  consisted  of  all 
TDWR  system  internal  interfaces  as  well  as  the  MPS  and  System 
Maintenance  Data  Terminal  (MDT)  external  interfaces  as  indicated 
in  figure  3.2-1.  The  MPS  interface  was  tested  using  an  MPS 
simulator.  Build  4  OT&E  was  conducted  using  several  iterations 
of  software  builds: 

a.  TAl  was  conducted  using  Build  4  s47,  s50,  s51,  and  s51a. 

b.  TA2  was  conducted  using  Build  4  s63. 

c.  TA3  was  conducted  using  Build  4  s63. 

3.2.2  BUILD  4+. 

The  Build  4+  OT&E  hardware  test  configuration  consisted  of  all 
TDWR  system  internal  Interfaces  as  well  as  the  LLWAS  II  backup 
communication  link  and  System  MDT  external  interfaces  as 
indicated  in  figure  3.2-1.  The  Build  4+  enhancement  provided  an 
interface  from  LLWAS  II  to  the  DFU  for  display  of  LLWAS  II  Center 
Field  (CF)  winds  on  the  RDTs.  An  anemometer  simulator  was  used 
at  the  CF  anemometer  to  simulate  wind  speed  and  direction.  Build 
4+  OT&E  was  conducted  using  two  software  builds:  TA4  was 
conducted  using  Build  4+  s67  and  s68. 

3.2.3  BUILD  5A. 

The  Build  5A  OT&E  hardware  test  configuration  consisted  of  all 
TDWR  system  internal  Interfaces  as  well  as  the  LLWAS  II 
primary/backup  communication  and  System  MDT  external  interfaces 
as  indicated  in  figure  3.2-1.  The  Build  5A  enhancement  added  an 
external  interface  from  LLWAS  II  to  the  RPG  and  provided  for 
display  of  LLWAS  II  boundary  and  CF  winds  on  the  GSDs  and  RDTs. 

An  anemometer  simulator  was  used  at  the  CF  anemometer  and  two 
other  anemometers  to  simulate  wind  speed  and  direction.  The  test 
configuration  also  included  additional  hardware  and  software  to 
provide  an  interface  to  the  Integrated  Terminal  Weather  System 
(ITWS).  This  interface  was  provided  for  the  ITWS  Demonstration 
and  was  not  tested  by  ACT-320.  Build  5A  OT&E  was  conducted  using 
two  software  builds: 

a.  TA5  was  conducted  using  Build  5A  sl5  (ITWS  version). 

b.  TA6  was  conducted  using  Build  5A  sl9  (ITWS  version). 


5 


HAS 


TDWR 

System 


Base  Data 
Recorder 


1  MPS  '  1 

!  1 

1 

MDT 

> 

<  > 

Onsmandsand 
Maintenance  Data 

> 

r  > 

_ 

RPG/RMS 


Ftoducte^Qmmu&ds 


Friaary  Connninncifoa  ^ 


(LLWASProducti) 


Piodiictat/CbotxDl  ^ 


Base  Data 
and  Product 
Display 


Archive 

Recorder 


Productc/Caatrol  ^ 


Display  Functional  Unit 
TRACON  Situation  and 
Ribbon  Displays 


Display  Functional  Unit 
J  Tower  Situation  and  Ribbon  k- 


LLWASn 

or 

LLWASffl 


Notes; 

1  OT&E  pcffoimcd  by  ACr-330 
^  Not  available  in  the  NAS 

3  iiiis  interface  was  tested  during  Builds  4, 4+,  and  5A  OT it  does  not  exist  in  the  Build  5B  coofiguiation 
^  This  interface  was  tested  during  Builds  4+,  5 A,  and  SB  OT&E 
^  Ibis  interface  was  tested  during  Builds  SA  and  SB  OT&E 
6  iius  interface  was  tested  during  BuOd  SB  OT&E 
RDT  status  is  used  by  LLWASm  while  in  backup  mode  only 

FIGURE  3.2-1.  TDWR  EXTERNAL  INTERFACES 
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3.2.4  BUILD  5B. 


The  Build  5B  OT&E  hardware  test  configuration  consisted  of  all 
TDWR  system  internal  interfaces  as  well  as  the  LLWAS  II  or  LLWAS 
III  primary/backup  communication  and  System  MDT  external 
interfaces  as  indicated  in  figure  3.2-1.  The  Build  5B 
enhancement  added  an  external  Interface  from  LLWAS  III  to  the  RPG 
and  provided  for  display  of  LLWAS  III  threshold  and  CF  winds  on 
the  GSDs  and  RDTs.  In  addition,  LLWAS  microburst  and  wind-shear 
alerts  are  integrated  with  TDWR  microburst  and  wind-shear  alerts 
to  produce  an  integrated  alert  for  display  on  the  GSDs  and  RDTs. 
The  Build  5B  enhancement  also  affected  an  internal  interface  by 
moving  the  Archive  Recording  function  from  the  RPG  to  the  DFU. 
Build  5B  OT&E  was  conducted  using  the  same  software  build:  TA7, 
TAB  TA9,  and  TAIO  were  conducted  using  Build  5B  sl5. 

3 . 3  INTERFACES . 

TDWR  provides  external  interfaces  to  LLWAS  II  and/or  LLWAS  III, 
RMMS,  MDT,  and  TCCC. 
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3.3.1  TDWR- -LLWAS  I I . 


The  TDWR  to  LLWAS  II  communications  interface  consists  of  a 
primary  and  backup  interface,  and  is  described  in  NAS-IR- 
31023105,  Part  2,  Revision  A.  The  primary  interface  connects  the 
TDWR  RPG  to  the  LLWAS  Tower  Display  port  on  the  LLWAS  processor. 
The  backup  interface  connects  the  TDWR  Tower  GSD  to  the  LLWAS 
Spare  port  on  the  LLWAS  processor.  In  the  event  of  a  primary 
interface  failure,  the  switchover  from  primary  to  backup  occurs 
automatically.  Both  interfaces  operate  at  1200  bits  per  second 
(bps).  Figure  3.3. 1-1  presents  the  TDWR — LLWAS  II  interface. 
Note:  TE  stands  for  transmission  equipment. 


LLWAS  Point  of 


FIGURE  3.3.1 -1.  TDWR-LLWASII  INTERFACE 
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TDWR--LLWAS  III. 


The  TDWR  to  LLWAS  III  communications  interface  consists  of  a 
primary  and  backup  interface,  and  is  described  in  NAS-IR- 
31023105,  Part  1,  Revision  C.  The  primaiy  interface  connects  the 
TDWR  RPG  to  the  LLWAS  processor.  The  backup  interface  connects 
the  TDWR  Tower  GSD  to  the  LLWAS  processor.  In  the  event  of  a 
TDWR  failure,  the  switchover  from  primary  to  backup  occurs  after 
active  runways  are  configured  using  the  LLWAS  keypad.  The  GSD 
operator  then  acknowledges  the  switchover  to  the  backup  link  via 
a  GSD  menu  indicating  that  TDWR  is  nonoperational .  Both 
interfaces  operate  at  9600  bps.  Figure  3. 3. 2-1  presents  the 
TDWR — LLWAS  III  interface. 


LLWAS  Point  of 


FIGURE  3.3.2-1 .  TDWR-LLWAS  III  INTERFACE 
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3.3.3  TDWR— RMMS. 


The  TDWR  RPG/RMS  provides  the  following  interfaces: 

a.  RMMS;  The  RMMS  utilizes  the  MPS  to  remotely  control 
and  monitor  the  TDWR.  This  interface  is  described  in  NAS-MD-790 
and  NAS-MD-793. 

b.  MDT:  The  MDT  is  located  at  the  TDWR  site  in  the 
Concrete  Masonry  Unit  (CMU)  and  is  used  to  locally  control  and 
monitor  the  TDWR.  This  interface  can  operate  at  1200,  2400, 
4800,  and  9600  bps. 

These  interfaces  are  presented  in  figure  3. 3. 3-1. 


FIGURE  3.3.3-1.  TDWR-MPS  AND  TDWR-MDT  INTERFACES 
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4.  TEST  AND  EVALUATION  DESCRIPTION. 


Detailed  descriptions  of  test  subelements,  objectives, 
participants,  and  methodologies  are  contained  in  appendix  B. 

4.1  TEST  SCHEDULE  AND  LOCATIONS. 

Appendix  A,  table  A-1  provides  a  summary  of  all  test  activities 
with  dates  and  locations . 

4.2  PARTICIPANTS. 

Appendix  A,  table  A-2  provides  a  summary  of  all  participating 
organizations  and  their  functional  contributions. 

ACT-320  acted  as  the  agent  of  the  Program  Manager  to  manage  the 
Test  and  Evaluation  (T&E)  program  per  FAA  Order  1810.4B. 
Specifically,  ACT-320  performed  the  following  OT&E-related 
activities : 

a.  Prepared  OT&E  Integration  and  OT&E  Operational  test 
plans,  procedures,  and  reports; 

b.  Directed  and  conducted  all  OT&E  Integration  and  OT&E 
Operational  tests  and  retests; 

c.  Reviewed  AOS-250  OT&E  Shakedown  test  requirements, 
plans,  procedures,  and  reports; 

d.  Monitored  AOS-250  OT&E  Shakedown  tests  and  retests. 

Many  other  FAA  organizations  participated  in  OT&E  testing, 
providing  technical  knowledge,  maintenance  support,  test 
expertise,  etc.  These  organizations  and  their  activities  follow 

The  Radar  Support  Engineering  Branch,  AOS-250,  performed  the 
following  OT&E-related  activities: 

a.  Prepared  OT&E  Shakedown  requirements,  plans,  procedures 
and  reports; 

b.  Directed  and  conducted  all  OT&E  Shakedown  test  and 
retests; 


c.  Reviewed  OT&E  Integration  and  OT&E  Operational  test 
requirements,  plans,  and  reports; 

d.  Monitored  and  participated  in  OT&E  Integration  and  OT&E 
Operational  tests  and  retests. 

The  Communications  Infrastructure  Branch,  ACT-330,  prepared  TDWR 
-RMS  OT&E  Integration  test  procedures,  directed  and  conducted 
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TDWR — RMS  OT&E  Integration  tests  and  retests  and  prepared  the 
appropriate  reports .  This  organization  also  performed  Lightning 
Protection  and  Grounding  Surveys  on  the  Houston  and  Oklahoma  City 
sites  to  verify  compliance  to  FAA-STD-019b. 

AOS-220  participated  in  the  Build  5A  and  Build  5B  OT&E 
Integration  and  OT&E  Operational  test  conduct. 

AML-446  conducted  bullgear  wear  and  vibration  analyses. 

AMA-441  provided  support  during  OT&E  Operational  retests. 

The  Civil  Aviation  Security  Division,  ASO-700,  conducted  a 
comprehensive  security  risk  assessment,  and  developed  the 
baseline  Sensitive  Application  Certification  (SAC)  and 
Accreditation  Document  for  the  TDWR  system. 

Southern,  Southwest,  Eastern,  and  Central  Regional  Airway 
Facilities  Divisions,  as  well  as  the  Memphis,  Houston,  St.  Louis 
Airway  Facilities  Sectors  participated  in  OT&E  Integration,  OT&E 
Operational,  and  OT&E  Shakedown  test  conduct. 

Southern  and  Southwest  Regional  Air  Traffic  Divisions,  as  well  as 
the  Memphis  and  Houston  Air  Traffic  Facilities  participated  in 
OT&E  Integration,  OT&E  Operational,  and  OT&E  Shakedown  test 
conduct . 

The  National  Severe  Storms  Laboratory  (NSSL),  under  contract  by 
ACT-320  and  the  Program  Office,  provided  extensive  meteorological 
and  data  analysis  support  during  all  OT&E  Operational  Weather 
Performance  Tests. 

4.3  TEST  AND  SPECIALIZED  EQUIPMENT. 

The  following  test  equipment  was  used  during  the  course  of  this 
testing: 

MPS  Simulator 
Anemometer  Simulator 

LLWAS  Scenario  Tapes  (For  LLWAS — TDWR  interface  testing) 
Weather  Scenario  Test  Tape  (AOS-250) 

Power  Monitor,  BMI  4800 
Protocol  Analyzer,  HP  4957A 
Oscilloscope,  Tektronics,  2340A 
Spectrum  Analyzer 
Signal  Generators,  Gigatronics 

NSSL  Supplied  Weather  Scenario  Tapes  Used  For  Weather 
Product  Truthing 
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4.4  TEST  OBJECTIVES/CRITERIA. 


Detailed  descriptions  of  test  objectives  are  provided  in  appendix 

B. 


4.4.1  OT&E  Integration. 

OT&E  Integration  consisted  of  testing  NAS  System  End-to-End 
Performance.  Specifically,  NAS-SS-1000  Volume  I  System  Level, 
and  Volumes  II  through  V,  Subsystem  Level  Requirements,  as 
identified  in  the  Baseline  TDWR  OT&E  Test  Plan  and  the  TDWR  Build 
5  TEMP,  were  tested.  This  testing  established  NAS  baseline 
performance  or  verified  that  previously  existing  NAS  performance 
was  not  degraded. 

The  OT&E  Integration  effort  was  conducted  with  the  following 
objectives : 

a.  Verify  the  TDWR's  capability  to  properly  interface  and 
function  with  the  associated  NAS  subsystems,  including  hardware, 
software,  operational  and  maintenance  activities; 

b.  Verify  that  the  interfaces  support  the  specification's 
mandated  performance; 

c.  Ensure  the  early  detection  of  interface  design  problems; 

d.  Minimize  site  problems  by  comprehensive  integration 
testing  and  evaluation; 

e.  Collection  of  system  reliability  and  safety 
measurements ; 

f.  Verify  the  requirements  of  NAS  System  Specification. 

4.4.2  OT&E  Operational. 

OT&E  Operational  testing  had  the  intent  of  verifying  the 
operational  effectiveness  and  suitability  of  the  equipment  with 
user  participation  in  the  evaluation  testing.  Aspects  of  OT&E 
Operational  testing  are  as  follows; 

a.  Reliability,  maintainability,  and  availability; 

b.  Degraded  operations  and  operational  utilization 
scenarios ; 

c.  Stress  and  NAS  loading  testing  of  all  interoperable 
subsystems ; 
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d.  Human  factors; 

e.  Safety  and  security; 

f.  Site-adaptation; 

g.  Transition  switchover. 

OT&E  Operational  testing  employed  system  users  to  assess 
operational  suitability  and  effectiveness  of  the  subsystem  in  the 
NAS  environment.  The  OT&E  Operational  testing  effort  was 
conducted  with  the  following  objectives: 

a.  Verification  and  validation  of  the  operational 
requirements ; 

b.  Verification  and  validation  of  operational  and 
maintenance  procedures; 

c.  Verification  and  validation  of  system  documentation 
completeness  and  useabllity; 

d.  Evaluation  of  the  effectiveness  of  contractor-developed 
training  programs; 

e.  Evaluation  and  determination  of  the  effect  of  the 
segment  under  test  on  the  operational  mission; 

f.  Identification  and  evaluation  of  the  safety  factors 
involved  during  transition  and  determination  that  transition  can 
be  achieved  safely; 

g.  Evaluation  of  the  subsystem's  operations  and  maintenance 
with  respect  to  the  variations  in  site  configurations  and 
adaptation; 

h.  Assessment  of  the  subsystem's  capability  to  support 
current  and  future  modifications; 

i.  Assessment  and  evaluation  of  the  readiness  of  personnel 
and  procedures  for  field  deployment  and  operational  use. 

4. 4. 2.1  Reliability.  Maintainability,  and  Availability  (RMA) . 

The  RMA  T&E  was  conducted  to  estimate  and  verify  that  the  RMA 
requirements  (parts  accessibility.  Mean  Time  Between  Failures 
(MTBF),  Mean  Time  Between  Critical  Failures  (MTBCF),  etc.)  were 
achievable  in  an  operational  environment. 

4. 4. 2. 2  Degraded  Operations. 

This  test  was  conducted  to  determine  the  acceptability  of  the 
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resultant  operational  degradation  when  failures  are  induced  in 
the  system. 

4. 4. 2. 3  Stress  and  NAS  Loading. 

This  test  was  conducted  to  estimate  and  determine  the  levels  of 
stress  and  NAS  loading  provided  by  the  operational  environment. 

4 . 4 . 2 . 4  Human  Factors . 

This  test  was  conducted  to  evaluate  the  interaction  of  personnel 
with  the  system  in  the  operational  environment. 

4. 4. 2. 5  Safety. 

This  test  was  conducted  to  estimate  and  determine  the  degree  to 
which  the  system  met  personnel  safety  requirements . 

4. 4. 2. 6  Site  Adaptation. 

This  test  was  conducted  to  ensure  that  site  data  unique  to  each 
TDWR  facility  was  correctly  developed/  updated,  and  installed  in 
the  system. 

4. 4. 2. 7  Security. 

This  test  was  conducted  to  estimate  and  determine  the 
effectiveness  of  the  system  in  allowing  only  authorized  use. 

The  objectives  of  the  ASO-700  Security  Assessment  were  to  develop 
the  baseline  SAC  for  the  TDWR,  to  evaluate  and  develop  standard 
practices  and  procedures  for  required  security  controls  and 
countermeasures,  and  to  ensure  compliance  with  Deployment 
Readiness  Review  (DRR)  issues. 

4. 4. 2. 8  Transition  Switchover. 

This  test  was  conducted  to  estimate  and  determine  that  the  system 
and  procedures  were  such  that  a  move  from  the  old  system  (LLWAS) 
to  the  new  system  (integrated  TDWR — LLWAS)  and  vice  versa  could 
be  accomplished  without  degrading  NAS  operations  while  minimizing 
impact  on  the  user. 

4. 4. 2. 9  Weather  Performance. 

The  objective  of  the  Weather  Performance  test  is  to  verify  that 
TDWR  meets  operational  suitability  and  reliability  requirements 
in  relation  to  its  ability  to  detect  hazardous  weather. 

4.5  TESTING  DESCRIPTIONS. 

Appendix  A,  table  A-3  provides  traceability  of  functional 
subelements  to  Test  Activity. 
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4.6  DATA  COLLECTION  AND  ANALYSIS  METHOD. 


Pretest  procedures  were  written  in  accordance  with  baseline 
system  configuration,  and  were  modified  accordingly  when 
incremental  software  builds  were  installed  and  hardware 
modifications  were  retrofitted.  Scenario  Data  Tapes  were 
generated  and  were  played  back  to  ensure  the  system  reacted  as 
predicted. 

Weather  data  were  collected  by  NSSL  for  algorithmic  verification. 
Data  were  analyzed  on  real-time  displays  or  collected  on  tape  or 
disk  for  further  off-line  analysis.  Pictures  were  taken  (e.g., 
transmitter  spectrum)  to  analyze  the  integrity  of  the  system.  An 
AC  Power  Monitoring  Meter  with  real-time  graphic  printouts  was 
used  to  monitor  the  commercial  power  supply  to  the  TDWR  system. 

An  Investigative  Panel  of  meteorological  experts  was  assembled  to 
analyze  results  and  determine  the  best  ways  to  improve  system 
detection  capability  and  provide  weather  data  "truthing." 

More  specific  data  regarding  each  test  is  found  in  appendix  B. 


5.  RESULTS  AND  DISCUSSION. 

Appendix  B  provides  a  test  description  and  summary  of  results  for 
each  of  the  OT&E  Subelements. 


6.  CONCLUSIONS. 

The  following  significant  problems  need  to  be  addressed: 

6.1  RELIABILITY. 

Early  testing  of  the  Terminal  Doppler  Weather  Radar  (TDWR) 
revealed  a  serious  reliability  problem.  The  installation  of  a 
Reliability  Enhancement  Package  in  January  94  provided 
substantial  improvement,  however,  the  reliability  re-test  (TA5) 
which  was  conducted  February  through  March  94  did  not  demonstrate 
the  required  level  of  reliability.  (See  appendix  B.)  In  spite  of 
the  fact  that  the  Harris  computer  at  Memphis  was  completely 
replaced,  there  have  been  subsequent  failures  of  Harris  boards  at 
Memphis.  There  has  been  an  unusually  high  number  of  Antenna 
Drive  Motor  failures.  While  modifications  since  that  time  have 
demonstrated  a  gradual  improvement,  there  has  been  no  further 
objective  study  which  would  Identify  the  actual  reliability 
numbers  for  the  TDWR. 
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6.2  Low-Level  Wind  Shear  fLLWAS)  II. 


The  LLWAS  II  mapping  issue  as  described  in  Service  Report  (SR) 
IAH94-SR001.  There  is  a  discrepancy  between  what  the  LLWAS 
processor  actually  transmits  versus  what  the  TDWR  expects  to 
receive.  The  LLWAS  II  transmits  a  Center  Field  Wind  (30-second 
average)  in  a  field  in  which  the  TDWR  is  expecting  sensor  data. 
This  problem  manifests  itself  differently  dependent  upon  the 
local  LLWAS  configuration  file. 

6.3  Low-Level  Wind  Shear  (LLWAS)  III. 

TDWR — LLWAS  primary  link  problems  persist  even  though  substantial 
improvements  have  been  made  in  this  area.  Periodically/  a  good 
information  frame  is  transmitted  from  the  LLWAS  master  station 
through  the  RS-232  to  RS-530  converter  to  the  modem.  This  frame 
is  either  corrupted  in  the  LLWAS  modem,  leased  line,  TDWR  modem, 
or  a  combination  thereof  because  when  it  reaches  the  TDWR 
junction  box  it  has  been  transformed  into  an  aborted  frame.  This 
problem  was  observed  in  Denver. 

6.4  SAFETY. 

The  AMH-400  Job  Safety  Analysis  indicated  potential  safety 
deficiencies  in  pedestal-related  fall  hazards.  One  of  these  is 
the  absence  of  pedestal  ladder  safety  climb  devices.  (See 
appendix  B. ) 

6.5  WEATHER- RELATED  ISSUES. 

As  indicated  by  the  Investigative  Panel  (see  appendix  B),  the 
weather-related  issues  are: 

a.  Gust  Front  Misses 

b.  Gust  Front  False  Alarms 

c.  Microburst  False  Alarms 

6.6  ADAPTATION. 

There  is  a  concern  regarding  the  verification  of  adaptation  data. 
With  the  exception  of  Houston,  there  was  no  objective  data 
available  to  check  the  adaptation  parameters  against.  This  makes 
it  nearly  impossible  to  verify  whether  or  not  these  parameters 
are  correct. 

6 . 7  SERVICE  REPORTS . 

There  are  a  number  of  SRs  which  still  remain  open.  See  tables  A4 
and  A5.  The  SRs  listed  in  table  A5  do  not  include  those  AOS-250 
and  ACT-330  SRs  which  still  remain  open. 
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7 .  RECOMMENDATIONS . 


7.1  RELIABILITY. 

AND-420  and  ALM-400C  have  both  recognized  the  problem  with  the 
reliability  of  the  antenna  drive  motors.  A  number  of  possible 
improvements  are  being  or  have  been  identified.  Testing  of  some 
of  these  options  is  being  performed  now  and  more  is  planned. 

This  effort  will  result  in  a  set  of  alternative  solutions  being 
defined.  Selection  and  implementation  of  an  appropriate  solution 
will  be  based  on  cost  effectiveness  and  other  parameters. 

7.2  Low-Level  Wind  Shear  Alert  System  (LLWAS^  II. 

This  interface  is  not  in  accordance  with  the  Interface 
Requirements  Document  (IRD)  NAS-IR-31023105  Part  2,  Revision  A 
dated  December  2,  1992.  (Reference  SR  IAH94-0001).  If  the 
interface  or  the  corresponding  documentation  is  not  be  changed, 
ACT- 320  recommends  procedures  be  developed  to  ensure  sensor 
mapping  is  correctly  implemented  at  all  Terminal  Doppler  Weather 
Radar  (TDWR)/LLWAS  II  sites. 


7.3  Low-Level  Wind  Shear  Alert  System  (LLWAS)  III. 

Previously  observed  link  problems  have  been  substantially  reduced 
with  the  implementation  of  new  software  releases .  The  problem 
with  aborted  frames  remains.  AOS-220  has  protocol  analyzer  data 
and  is  working  on  the  issue.  It  may  be  that  the  modem  parameters 
will  have  to  be  optimized. 


7 . 4  SAFETY . 

A  contract  has  been  awarded  to  correct  safety  deficiencies.  A 
prototype  platform  is  expected  to  be  demonstrated  in  the  Fall  of 
1995  with  a  first  article  being  delivered  in  early  1996. 


7.5  WEATHER-RELATED  ISSUES. 

Solutions  recommended  by  the  Investigative  Panel  for  Weather- 
Related  issues  are  as  follows: 

a.  Gust  Front  Misses.  The  majority  of  gust  front  misses 
were  caused  by  multiple  trip  echoes .  The  Investigative  Panel 
feels  that  continued  long-term  research  may  result  in  more  robust 
range  de-obscuration  editing  techniques . 

b.  Gust  Front  False  Alarms.  Most  of  the  false  gust  front 
detections  were  due  to  dealiasing  errors.  Better  dealiasing 
techniques  are  needed.  Also,  there  were  several  false  detections 
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in  the  presence  of  vertical  wind  shear.  A  possible  remedy  is  to 
add  a  technique  to  the  algorithm  to  recognize  and  remove  the 
false  alarms  based  on  the  knowledge  that  vertical  wind  shear  is 
present. 


c.  Microburst  False  Alarms.  There  is  concern  that  there 
are  too  many  microburst  false  alarms  (18  percent  False  Alarm  Rate 
(FAR)  exceeds  the  performance  design  goal).  Results  indicate 
that  many  of  the  microburst  false  alarms  were  caused  by  spurious 
velocity  data  in  low  reflectivity,  divergent  regions  away  from 
storm  cores.  ACT-320  and  National  Severe  Storms  Laboratory 
(NSSL)  recommend  the  following  remedies: 

1.  Velocity  point-target  editing:  Many  of  the 
spurious  data  points  may  be  eliminated  by  the 
implementation  of  this  technique. 

2.  Implement  storm  cell/microburst  shape  overlap 

test:  This  would  require  microburst  detections  to 

be  more  closely  associated  with  a  storm  cell. 

3.  Reduce  storm  cell  test  to  0  kilometer  (km):  This 
would  also  require  microburst  detections  to  be 
more  closely  associated  with  a  storm  cell. 

4.  Add  reflectivity  thresholding:  Many  of  the  false 
alarms  were  in  areas  of  weak  reflectivity. 

5.  Raise  microburst  elevation  angle:  If  clutter  are 
indeed  affecting  the  data  on  the  microburst  tilt, 

a  0.1'  to  0.2'  increase  in  the  elevation  angle  may 
reduce  those  effects.  (The  MTS  location  should  be 
adjusted  accordingly. ) 

6.  Increase  the  signal-to-noise  ratio  threshold  for 
the  shear  segments:  The  same  reasoning  for 
reflectivity  thresholding  applies  here. 

7 .  Increase  the  delta-v  threshold  and  the  number  of 
shear  segments  required:  Although  it  is  not 
certain,  it  is  believed  that  many  of  the  false 
alarms  barely  exceeded  these  thresholds. 

7.6  ADAPTATION. 

ACT-320  recommends  that  all  adaptation  data  provided  to  Raytheon 
be  captured  in  a  database  and  used  to  verify  site  parameters  upon 
site  acceptance.  This  would  enable  early  detection  of  errors  and 
at  least  ensure  that  the  parameters  provided  to  the  contractor 
have  been  implemented  correctly. 
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7.7  SERVICE  REPORTS  fSRK 


ACT-320  recommends  AND-420  develop  a  plan  for  the  closure  of  all 
remaining  SRs. 


8.  ACRONYMS  AND  ABBREVIATIONS. 


ADR 

AF 

APD 

ARENA 

AT 

ATC 

ATCS 


Archive  Data  Recorder 

Airway  Facilities 

Archive  Playback  Device 

Area  Noted  For  Attention 

Air  Traffic 

Air  Traffic  Control 

Air  Traffic  Control  Specialist 


BDD 

BDR 

bps 


Base  Data  Display 
Base  Data  Recorder 
Bits  Per  Second 


CF  Center  Field 

CFW  Center  Field  Wind 

CMU  Concrete  Masonry  Unit 

CREM  Clutter  Residue  Editing  Map 

DFU  Display  Functional  Unit 

DRR  Deployment  Readiness  Review 

DTE  Design,  Test  &  Evaluation 


FAA 

FAR 


Federal  Aviation  Administration 
False  Alairm  Ratio 


GF  Gust  Front 

GFDA  Gust  Front  Detection  Algorithm 

GSD  Geographic  Situation  Display 


ICD 

IP 

IRD 

ITWS 


Interface  Control  Document 
Investigative  Panel 
Interface  Requirements  Document 
Integrated  Terminal  Weather  System 


km 


kilometer 


LL 

LLWAS 
LLWAS  II 
LLWAS  III 
LRU 


Lincoln  Laboratory 
Low-Level  Wind  Shear  Alert  System 
Low-Level  Wind  Shear  Alert  System  Phase  II 
Low-Level  Wind  Shear  Alert  System  Phase  III 
Line  Replaceable  Unit 


MBA  Microburst  Alert 

MDA  Microburst  Detection  Algorithm 

MDT  Maintenance  Data  Terminal 
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MIT 

MTBCF 

MTBF 

MTP 

NAS 

NCAR 

NSSL 

OSHA 

OT&E 

POD 


RDA 

RDT 

RMA 

RMMS 

RMS 

RPG 

SAC 

SR 

TCCC 

TDWR 

TE 

T&E 

TIB 

TRACON 

TVRTM 

WSA 


Massachusetts  Institute  of  Technology 
Mean  Time  Between  Critical  Failures 
Mean  Time  Between  Failures 
Master  Test  Plan 

National  Airspace  System 

National  Center  for  Atmospheric  Research 

National  Severe  Storms  Laboratory 

Occupational  Safety  and  Health  Administration 
Operational  Test  and  Evaluation 

Probability  of  Detection 


Radar  Data  Acquisition 
Ribbon  Display  Terminal 

Reliability,  Maintainability  and  Availability 
Remote  Maintenance  Monitoring  System 
Remote  Monitoring  Subsystem 
Radar  Products  Generator 

Sensitive  Application  Certification 
Service  Report 

Terminal  Control  Computer  Complex 
Terminal  Doppler  Weather  Radar 
Transmission  Equipment 
Test  and  Evaluation 
Technical  Instruction  Book 
Terminal  Radar  Approach  Control 

Test  Verification  Requirements  Traceability  Matrix 
Wind  Shear  Alert 
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APPENDIX  A 


REFERENCE  DATA 


Table  A-1.  TDWR  Test  Activities 
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ORGANIZATION  FUNCTION  TEST 

ACTIVITY 


,-2 


NSSL  Meteorological  Support  &  Data  Analysis  TA1;  TA2;  TA7 


Table  A-3.  TEST  ACTIVITY-OT&E  SUB-ELEMENTS  TRACEABILITY  MATRIX 


,-3 


Table  A-4.  Service  Report  Status 


Table  A-5.  Open  Service  Reports 


9/20/95 


OPEN  SERVICE  REPORTS 


1  SERVICE 

1  REPORT 

ASSIGNEE 

DATE 

ASSIGNED 

PR 

SUBJECT 

pVX-SR014 

TECHNICAL  OFFICER 

2/8/95 

Re-Boot  ALARM  -  STR  #3763 

PVX-SR015 

TECHNICAL  OFFICER 

2/8/95 

LLWAS  Primary  Link  DropOuts 

pvx-sRoie 

TECHNICAL  OFFICER 

2/8/95 

Elimination  of  Wind  Shear  Alert 

DVX-SR017 

TECHNICAL  OFFICER 

2/8/95 

Center  Field  (CF)  Wind  Product  Not  Updating 

IAH94-SR001 

TECHNICAL  OFFICER 

5/17/94 

H 

Violation  of  NAS-IR-31023105,  LLWAS  ll/TDWR  Interface 

MEM93-SR102 

TECHNICAL  OFFICER 

12/20/93 

H 

Instability  Residue  Alarms 

MEM93-SR134 

TECHNICAL  OFFICER 

8/25/93 

History  Log 

MEM93-SR137 

TECHNICAL  OFFICER 

8/27/93 

Gust  Front  on  Tower  GSD 

MEM93-SR144 

TECHNICAL  OFFICER 

9/1/93 

HSA-Boot  ALARM 

MEM93-SR145 

TECHNICAL  OFFICER 

9/1/93 

Harris  Re-Boot  Capability 

MEM93-SR160 

TECHNICAL  OFFICER 

9/7/93  : 

DDC  Reset 

MEM94-SR002 

TECHNICAL  OFFICER 

2/7/94 

TIB  Changes 

MEM9isR007 

TECHNICAL  OFFICER 

3/7/94 

H 

Instability  Residue  ALARM 

MEM94-SR014 

TECHNICAL  OFFICER; 

3/18/94  i 

ETHERNET  Cable  Connectors 

MEM94-SR015 

TECHNICAL  OFFICER 

3/24/94 

8mm  Tape  Recorder 

MEM94-SR016 

TECHNICAL  OFFICER 

4/19/94 

Ribbon  Display  Terminal  #1 

MEM94-SR017 

TECHNICAL  OFFICER 

4/19/94 

H 

Loss  of  Center  Field  Wind  (CFW) 

MEM94-SR018 

TECHNICAL  OFFICER 

4/19/94 

H 

Loss  of  LLWAS  Wind  Information 

MEM94-SR019 

TECHNICAL  OFFICER 

5/11/94 

MEM  Site  Modems  to  Junction  Box  Connectivity 

MEM94-SR020 

TECHNICAL  OFFICER 

4/11/94  ; 

Memphis  Tower  and  TRACON  DFU  Runway  Configurations 

MEM94-SR025 

TECHNICAL  OFFICER 

4/20/94  ; 

GSD  Audible  Alarm 

MEM94-SR026 

TECHNICAL  OFFICER 

5/11/94 

Proper  Setting  of  Memphis  TDWR  Modems 

OKC-SR009 

TEST  DIRECTOR 

5/28/93 

Gust  Front  False  Alarms 

OKC-SROU 

TECHNICAL  Officer 

10/25/93 

Error  Reporting  (2806) 

OKC-SR017 

TECHNICAL  OFFICER 

8/17/93  j 

Removal  of  Range-Folded  Echoes 

OKC-SR019 

TEST  DIRECTOR 

3/11/93 

Gust  Front  Alarm 

OKC-SR022 

TEST  DIRECTOR 

2/17/93 

MBA  on  Runway  Corridor 

OKC^SR029 

TECHNICAL  OFFICER 

8/17/93 

False  Gust  Front  Detections 

OKC-SR058 

TESTpIRECTOR 

2/17/93 

Unrealistic  Microburst  Values  on  GSD  (2806) 

6kC-SR083 

TECHNICAL  OFFICER 

8/17/93 

Clutter  Residue  Editing  Maps 

OKC-SR113 

TECHNICAL  OFFICER 

1/14/93 

Storm  Cell  Check 

OKC93-SR019 

TECHNICAL  OFFICER 

8/12/93 

Microburst  False  Alarm  Ratio 

6kC93-SR020 

TECHNICAL  OFFICER 

8/12/93 

Gust  Front  Misses 

OKC93-SRP21 

TECHNICAL  OFFICER 

,  8/12/93 

Grey  Shading  on  the  GSD 

|oKC94-SR001 

TECHNICAL  OFFICER 

"  3/22/94 

Velocity  Dealiasing  Errors 
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APPENDIX  B 


TDWR  TEST  ACTIVITIES 


OT&E  Subelement:  NAS  Subsystem  Integration 

Test  Objective:  To  determine  that  the  TDWR — MDT,  TDWR — Base 

Data  Display  (BDD),  TDWR — Base  Data  Recorder 
(BDR),  and  TDWR — Archive  Data  Recorder  (ADR) 
interfaces  exist  and  are  functional  in 
accordance  with  the  appropriate  requirements . 

Test  Participants:  ACT-320;  AOS-250;  ASO-452;  Memphis,  Denver, 

Kansas  City,  and  Orlando  Air  Traffic  (AT)  and 
Airway  Facilities  (AF) 

Data  Collection  and 

Analysis  Method:  MDT  screens  and  BDD  graphic  base  product 

printouts  were  collected;  BDD  was  monitored 
for  accuracy.  Base  data  were  recorded  and 
played  back  on  the  BDR.  Archived  weather 
products  were  recorded  and  played  back  on  the 
ADR. 

Test  Description/Results : 

The  TDWR — MDT  interface  was  formally  tested 
in  Oklahoma  City  (TAl),  and  was  verified 
during  all  other  test  periods  primarily 
through  normal  daily  system  operation. 

The  test  team  verified  the  TDWR — MDT 
interface  and,  in  particular,  MDT  operation, 
by  reviewing  all  path  options  and  by 
initiating  and  verifying  system  commands, 
i.e.,  system  reconfiguration,  command 
diagnostic  tests,  power  off  antenna,  display 
ALARM  screen,  etc.  The  TDWR — MDT  interface 
was  found  to  operate  properly.  However,  the 
test  team  did  note  several  errors  in  the 
Technical  Instruction  Book  (TIB),  Software 
Users  Manual  for  the  RMS  Software,  and 
prepared  SRs  highlighting  these  errors.  See 
Maintainability  Test  Results  for  TIB  error 
resolution. 

Similarly,  the  TDWR — BDR  and  TDWR — BDD 
interfaces  were  formally  tested  in  Oklahoma 
City  (TAl),  and  were  verified  during  most 
other  test  periods  primarily  through  normal 
daily  system  operation.  The  BDR  was  used  for 
playback  of  TDWR  scenario  tapes. 

Enhancements  to  the  BDD  were  included  as 
part  of  the  Build  5B  enhancement,  and 
included  the  following:  zoom/unzoom, 
continuous  request,  time  lapse,  cursor 
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position  readout,  etc.  These  enhancements 
were  tested  in  Denver  (TA7)  and  Orlando 
(TAIO)  and  were  found  to  function  properly. 

The  TDWR--ADR  interface  was  also  included  as 
part  of  the  Build  5B  enhancement.  To  verify 
this  interface,  ACT- 320  downloaded  GSD 
archive  data  to  tape  using  the  ADR, 
transported  the  ADR  to  the  TDWR  site, 
connected  the  ADR  to  the  BDD  and  reconfigured 
the  BDD  as  an  archive  data  playback  device 
(APD),  transferred  the  archive  data  to  the 
APD  hard  disk,  and  played  back  the  archive 
files.  ACT-320  successfully  performed  this 
test  in  Denver  (TA7)  and  Orlando  (TAIO). 
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OT&E  Subelement:  NAS  System  Integration 

Test  Objective:  To  ensure  that  TDWR  external  interfaces 

(TCCC,  RMMS,  LLWAS)  exist  and  are  functional 
in  accordance  with  the  appropriate 
requirements . 

Test  Participants:  ACT-320;  AOS-220/250;  Memphis,  Houston, 

Denver,  Kansas  City,  and  Orlando  Air  Traffic 
and  Airway  Facilities 

Data  Collection  and 

Analysis  Method:  Protocol  analyzer  data  was  collected  to 

determine  if  interfaces  operated  properly; 
scenario  tapes  were  used,  displays  were 
monitored,  and  alarms/alerts  were  logged  to 
determine  if  the  TDWR  responded 
appropriately . 

Test  Description/Results: 

The  TCCC  is  not  available  and,  therefore,  the 
TDWR — TCCC  interface  was  not  tested. 

The  Communications  and  Infrastructure  Branch, 
ACT-330,  who  has  the  responsibility  for 
verifying  the  TDWR — RMMS  interface,  conducted 
baseline  and  Build  5  TDWR— RMMS  OT&E 
Integration  tests  and  retests  in  Oklahoma 
City  (1992),  Houston  (1993),  St.  Louis 
(1994),  and  Washington,  DC  (1995).  Test 
descriptions  and  results  are  highlighted  in 
ACT-330  test  reports. 

ACT-320  conducted  TDWR— LLWAS  II  tBuild  4+) 
OT&E  tests  in  Houston  (TA3)  to  verify  that 
LLWAS  centerfield  winds  were  accurately 
displayed  on  the  RDTs.  The  test  was 
conducted  using  both  simulated  and  real-time 
centerfield  wind  data.  In  each  case,  TDWR 
RDT  and  LLWAS  displays  were  monitored  for 
data  correlation.  In  both  the  simulated  and 
real-time  tests,  the  TDWR  RDT  and  LLWAS 
displays  showed  the  same  centerfield  wind 
data . 

ACT-320  conducted  TDWR— LLWAS  II  Build  5A 
OT&E  integration  tests  in  Houston  (to 
coincide  with  AOS-250  Build  5A  OT&E  Shakedown 
tests  while  the  remainder  of  ACT-320 's  Build 
5A  OT&E  took  place  in  Memphis  (TA6))  to 
verify  that  LLWAS  centerfield  and  sensor 
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winds  are  accurately  displayed  on  the  GSDs 
and  RDTs  and  to  satisfy  NAS-SS-1000 
requirements.  ACT-320  used  an  LLWAS 
anemometer  simulator  to  simulate  wind  speed 
and  direction  at  three  sensors  (centerfield 
and  sensors  4  and  5).  LLWAS  sensors  were 
mapped  to  selected  runways.  The  GSDs  and 
RDTs  were  monitored  for  display  accuracy  as 
the  wind  speed  and  direction  were  varied. 

Test  results  were  varied.  Centerfield  wind 
information  was  accurately  displayed  on  the 
GSDs  and  RDTs  and  the  four  NAS-SS-1000 
requirements  were  successfully  verified. 
However,  while  the  centerfield  wind 
information  was  accurately  displayed  on  the 
TDWR  displays,  other  sensors'  wind 
information  was  not  accurately  displayed.  A 
discrepancy  exists  between  the  data 
transmitted  from  the  LLWAS  II  processor  and 
the  data  expected  by  the  TDWR.  The  following 
table  shows  this  discrepancy  for  Houston: 


LLWAS  II 
Transmits 

TDWR  Expects 

CF  2- min  avg 

CF  2 -min  avg 

CF  30-sec  avg 

Sensor  2 

Sensor  2 

Sensor  3 

Sensor  3 

Sensor  4 

Sensor  4 

Sensor  5 

Sensor  6 

Sensor  6 

The  discrepancy  was  clear  when  sensors  2,  3, 

4 ,  and  5  were  mapped  to  runways .  The  wind 
information  for  sensor  2  was  actually  CF  30- 
second  average,  sensor  3  was  actually  sensor 
2  information,  sensor  4  was  actually  sensor  3 
information,  and  sensor  5  was  actually  sensor 
4  information.  Sensor  6  wind  information  was 
displayed  correctly. 

While  the  above  LLWAS  II  data  transmittal 
sequence  pertains  to  Houston,  all  LLWAS  II 
data  is  transmitted  accordingly;  CF  2-minute 
avg,  CF  30-second  average,  followed  by  four 
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other  sensors .  The  TDWR  expects  the  LLWAS 
data  as  in  the  table  above,  per  the  TDWR — 
LLWAS  Interface  Control  Document  (ICD),  dated 
August  1,  1993.  The  LLWAS  CF  30-second 
average  is  not  documented  in  the  ICD.  This 
discrepancy  between  transmitted  data  and 
expected  data  will  exist  at  all  LLWAS  II 
sites. 

ACT-320  conducted  TDWR— LLWAS  III 
configuration  checkout  tests  in  Denver  (TA7) 
to  verify  the  primary  and  backup  links  and  to 
verify  that  LLWAS  III  center field  and 
threshold  winds  and  microburst/wind-shear 
alerts  are  accurately  displayed  on  the  GSDs 
and  RDTs.  ACT-320  used  an  LLWAS  scenario 
tape  to  simulate  wind  data.  ACT-320  found 
the  backup  link  to  function  properly,  but 
observed  primary  link  dropouts  when  the  RDTs 
displayed  blanks  for  threshold  winds  and 
displayed  9s  for  centerfield  wind  data. 

These  drop-outs  lasted  15-20  seconds  and 
occurred  several  times  an  hour.  Effects  of 
the  TDWR — LLWAS  Integration  Algorithm  were 
evident;  LLWAS  microburst  and  wind  shear 
alerts  were  reduced  when  not  confirmed  by 
TDWR  and  vice  versa. 

ACT-320  conducted  the  TDWR— LLWAS  II  Build  5B 
OT&E  tests  in  Kansas  City  (TA8)  to  verify  the 
primary  and  backup  links  and  to  verify  that 
LLWAS  II  centerfield  and  sensor  threshold 
winds  were  accurately  displayed  on  the  GSDs 
and  RDTs.  Test  results  were  varied; 
however,  they  were  expected  since_  the  Build 
5B  LLWAS  II  interface  was  similar  to  the 
Build  5A  LLWAS  II  interface.  (Please  see 
Build  5A  OT&E  results  above.) 

ACT- 320  conducted  the  TDWR— LLWAS  III  Build 
5B  OT&E  test  in  Orlando  (TAIO)  to  verify  the 
primary  and  backup  links  and  to  verify  that 
LLWAS  III  centerfield  and  threshold  winds  and 
microburst/wind-shear  alerts  were  accurately 
displayed  on  the  GSDs  and  RDTs .  The  GSD  and 
RDTs  were  monitored  for  those  centerfield 
wind,  threshold  winds,  and  microburst  and 
wind  shear  alerts  present  on  the  LLWAS 
scenario  tape.  ACT-320  successfully 
completed  this  test  by  verifying  expected 
weather  phenomena  on  both  the  GSD  and  RDTs. 

As  expected,  effects  of  the  TDWR — LLWAS 
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Integration  Algorithm  were  evident;  LLWAS 
microburst  and  wind  shear  alerts  were  reduced 
when  not  confirmed  by  TDWR  and  vice  versa. 

The  following  examples  of  LLWAS  alerts  not 
confirmed  by  TDWR  were  seen:  "MBA  30K-"  was 
reduced  to  "WSA  25K-"  and  "WSA  15K-"  was 
reduced  to  no  alert. 

In  addition,  proper  operation  of  both  TDWR — 
LLWAS  primary  and  backup  links  were 
successfully  verified.  Analysis  of  the  data 
indicates  that  there  is  still  a  problem 
whereby  a  good  information  frame  is 
transmitted  from  the  LLWAS  master  station 
through  the  RS-232  to  RS-530  converter  to  the 
modem.  This  frame  is  either  corrupted  in  the 
LLWAS  modem,  leased  line,  TDWR  modem,  or  a 
combination  thereof  because  when  it  reaches 
the  TDWR  Junction  Box  it  has  been  transformed 
into  an  aborted  frame. 
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OT&E  Subelement:  NAS  End-to-End  Performance 

Test  Objective:  To  ensure  the  TDWR  system  can  operate  in  a 

full-up  configuration  (with  all  available 
internal  and  external  interfaces)  and  can 
provide  reliable,  useable,  and  suitable 
weather  products  to  controllers  in  a  timely 
manner . 


Test  Participants:  ACT-320;  AOS-250;  ASO-452;  Memphis,  Denver, 

Kansas  City,  and  Orlando  Air  Traffic  and 
Airway  Facilities 


Data  Collection  and 

Analysis  Method:  Monitored  system  performance,  logged  system 

failures  and  downtime. 

Test  Description/Results: 

During  OT&E  test  periods  in  Memphis  (TA4  to 
TA6),  Kansas  City  (TAB),  and  Orlando  (TAIO), 
ACT-320  allowed  the  system  to  operate  for 
extended  periods  of  time  with  all  internal 
interfaces  and  LLWAS  II  or  LLWAS  III.  ACT- 
320  monitored  system  performance  and  ensured 
the  TDWR  provided  useable  data  to  users. 

Detailed  test  results  related  to  end-to-end 
performance  are  highlighted  in  the  following 
tests:  NAS  System  Integration, 

Reliability /Availability,  Human  Factors  and 
Weather  Performance. 
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OT&E  Subelement: 


Reliability /Availability 


Test  Objective:  To  determine  that  the  TDWR  can  operate 

reliably  within  specification  and  be 
available  to  provide  weather  information. 

Test  Participants:  ACT-320;  AOS-250;  ASO-452;  ACE-452;  AAC-944; 

AML-426;  Memphis,  Houston,  Denver,  Kansas 
City,  and  Orlando  Air  Traffic  and  Airway 
Facilities;  NSSL 


Data  Collection  and 

Analysis  Method:  Record  operating  time  and  failure  data, 

including  downtime;  calculate  MTBF  and  MTBCF . 

Test  Description/Results: 

No  official  reliability  testing  was  performed 
in  Oklahoma  City  (TAl).  However,  during  the 
10-week  test  period,  the  TDWR  system 
performed  poorly  in  the  areas  of  system 
reconfiguration,  system  stability,  and 
power fail  recovery.  The  TDWR  system  also 
performed  poorly  in  Oklahoma  City  (TA2)  and 
Houston  (TA3).  ACT-320  therefore  felt  an 

official  system  reliability  test  was 
justified. 

ACT-320  monitored  system  reliability  over  the 
6-week  reliability  test  and  ATC  evaluation 
period  in  Memphis  (TA4).  In  particular,  ACT- 
320  kept  track  of  system  performance  and 
system  downtime  to  determine  TDWR  MTBF  and 
MTBCF.  MTBF  and  MTBCF  figures  for  the  6 -week 
period  were  63  hours  (specification  =  550 
hours)  and  112  hours  (specification  =  1500 
hours),  respectively.  An  antenna  motor 
failure  and  several  unwarranted  RPG/RMS 
Harris  computer  reboots  were  identified  as 
critical  reliability  issues. 

During  the  6-week  period,  sporadic  red  Xs 
(which  normally  indicate  the  TDWR  is  not  in 
an  operational  mode)  were  displayed  on  the 
tower  and  Terminal  Radar  Approach  Control 
(TRACON)  GSDs.  A  communication  link  problem 
between  the  TDWR  site  and  the  tower  building 
was  identified  to  be  the  cause  and  was 
subsequently  corrected. 

Also  during  the  6 -week  period,  the  tower  GSD 
displayed  a  Gust  Front  (GF)  over  the  airport 
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intersecting  all  active  runways.  The 
appropriate  GSD  ARENAS  were  highlighted  but 
no  RDT  alarm  was  issued.  Inspection  of  base 
data  showed  that  the  GF  should  have  sounded 
an  alarm  on  the  RDT.  This  underestimation  by 
the  Gust  Front  Detection  Algorithm  (GFDA)  may 
indicate  that  the  GFDA  may  need  refinement. 

Software  and  hardware  fixes,  designed  to 
Improve  system  reliability,  were  implemented 
prior  to  ACT-320 's  reliability  retest  in 
Memphis  (TA5).  During  the  4-week  retest 
period,  ACT-320  again  monitored  system 
performance  and  determined  the  MTBF  to  be  170 
hours  and  the  MTBCF  to  be  227  hours.  Of  the 
three  critical  failures,  two  were  RPG/RMS 
Harris  computer  reboots.  The  Memphis  Harris 
computer  had  historically  been  problematic 
and  was  replaced  in  May  1994.  ACT-320 

recognized  that  the  results  obtained  during 
this  retest  and  previous  tests  may  not  have 
been  representative  of  a  typical  TDWR  system. 

Finally,  ACT-320  monitored  the  TDWR  system 
during  Build  5A  and  Build  5B  OT&E  test 
periods  in  Memphis  {TA6),  Denver  (TA7  and 
TA9),  Kansas  City  (TAB),  and  Orlando  (TAIO) 
to  ensure  the  Build  5  enhancements  did  not 
degrade  system  reliability.  ACT-320 
determined  that  neither  Build  5A  nor  Build  5B 
degraded  system  reliability. 
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OT&E  Subelement:  Degraded  Operations 

Test  Objective:  To  determine  the  acceptability  of  the 

resultant  operational  degradation  when 
failures  are  induced  in  the  system. 

Test  Participants:  ACT-320;  AOS-250;  ASO-452;  Memphis,  Denver, 

Kansas  City,  and  Orlando  Air  Traffic  and 
Airway  Facilities 

Data  Collection  and 

Analysis  Method:  Monitored  the  MDT  to  verify  link  status, 

monitored  the  GSDs  and  RDTs  for  display 
verification . 

Test  Description/Results : 

ACT-320  successfully  conducted  this  test  in 
Memphis  (TA6)  during  Build  5A  OT&E.  ACT-320 
used  operational  scenarios  to  verify  TDWR 
response  to  TDWR — LLWAS  failure  modes;  these 
scenarios  included  the  following: 

a.  TDWR  and  LLWAS  operational, 

b.  TDWR  operational  and  LLWAS  non- 
operational, 

c.  TDWR  non-operational  and  LLWAS 
operational, 

d.  TDWR  and  LLWAS  non-operational . 

The  TDWR  was  put  in  maintenance  mode  to 
produce  the  "TDWR  non-operational"  state. 

The  LLWAS — RPG  and  LLWAS — DFU  connections 
were  interrupted  to  produce  a  "LLWAS  non- 
operational"  state.  In  Denver  and  Orlando 
(TA7,  TA9,  TAIO),  LLWAS  III  was  put  in  system 
support  state  to  simulate  a  "LLWAS  non- 
operational"  state. 

This  test  passed;  the  system  responded 
properly  to  TDWR — LLWAS  failure  scenarios  and 
link  communication  interruptions.  It  should 
be  noted  however,  that  should  the  LLWAS — TDWR 
primary  link  fail,  LLWAS  wind  information 
will  be  lost. 

ACT-320  successfully  conducted  this  test  in 
Denver  (TA7)  and  Kansas  City  (TA8)  using  the 
aforementioned  scenarios.  The  system 
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responded  properly  to  TDWR — LLWAS  failure 
scenarios  and  link  conununication 
interruptions.  In  particular,  in  Denver,  no 
RDT  rebuilding  problems  were  observed  while 
operating  under  the  backup  link,  as  had 
previously  been  reported  during  on-site 
Design  Test  and  Evaluation  (DT&E) . 

ACT-320  again  successfully  conducted  this 
test  in  Denver  (TA9)  using  the  aforementioned 
operational  scenarios.  The  system  responded 
properly  to  TDWR — LLWAS  failure  scenarios  and 
link  communication  interruptions.  In 
particular,  ACT-320  successfully  verified 
that  9s  were  indeed  displayed  on  the  GSDs  and 
RDTs  when  the  LLWAS  link  time-out  period 
exceeded  the  maximum  limit  (45  seconds). 
ACT-320  also  successfully  verified  proper 
back-up  link  operation  during  these  degraded 
scenarios . 

Finally,  ACT-320  successfully  conducted  this 
test  in  Orlando  (TAIO)  during  Build  5B  OT&E. 
The  system  responded  properly  to  TDWR — LLWAS 
failure  scenarios  and  link  communication 
interruptions.  However,  the  programmable 
alarm  timeouts  were  not  verified  because  the 
Orlando  TDWR  configuration  did  not  include 
tower  and  TRACON  audible  alarm  boxes. 

However,  these  alarm  timeouts  were  previously 
verified  in  Denver  (TA7). 
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OT&E  Subelement:  Stress  and  NAS  Loading 

Test  Objective:  To  estimate  or  determine  the  levels  of  stress 

and  NAS  loading  provided  by  the  operational 
environment . 

Test  Participants:  ACT-320;  AOS-250;  Memphis,  Denver,  and 

Orlando  Air  Traffic  and  Airway  Facilities 

Data  Collection  and 

Analysis  Method:  Monitored  GSDs  and  RDTs  to  verify  display 

accuracy  and  monitored  MDT  to  monitor  system 
status . 

Test  Description/Results : 

ACT- 320  conducted  this  test  during  the  Build 
5A  OT&E  in  Memphis  (TA6).  ACT-320  used  a 

weather  scenario  test  tape,  developed  by  AOS- 
250,  to  stress  the  TDWR  system.  This  test 
tape  consisted  of  level  6  precipitation,  wind 
shear,  microbursts,  and  gust  fronts.  ACT-320 
played  the  TDWR  scenario  test  tape  and 
monitored  the  TRACON  and  tower  GSDs  and  RDTs 
for  display  and  alarm/alert  accuracy.  (LLWAS 
was  in  a  live  state  because  there  was  no 
capability  to  run  a  scenario  tape  in  LLWAS  II 
systems.)  All  10  lines  of  each  RDT  were  used 
to  display  products  to  further  stress  the 
TDWR  system. 

The  test  was  successfully  completed;  the 
expected  weather  phenomena  and  alerts/alarms 
messages  were  accurately  displayed  on  the 
GSDs  and  RDTs . 

ACT-320  also  conducted  this  test  in  Denver 
(TA7).  Again,  ACT-320  used  a  weather 
scenario  test  tape,  developed  by  AOS-250,  to 
stress  the  TDWR  system.  ACT-320  played  the 
TDWR  and  LLWAS  weather  scenario  test  tapes, 
and  monitored  the  TRACON  and  tower  GSDs  and 
RDTs  for  display  and  alarm/alert  accuracy. 
During  test  conduct,  the  CF  wind  product 
(displayed  on  line  10  of  each  RDT)  did  not 
update  even  though  the  time  (also  on  line  10) 
and  all  RDT  threshold  winds  updated  properly. 
(The  CF  wind  product  displayed  a  constant  180 
10G35.)  ACT-320  LLWAS  personnel  identified 

this  as  a  procedural  problem  associated  with 
a  switch  to  the  backup  link.  Denver  AT  and 
AF  personnel  are  aware  of  this  problem  and 
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have  procedures  in  place  to  minimize  its 
impact.  In  addition,  ACT-320  observed  a  3- 
to  4 -second  delay  between  the  time  an 
expected  alert  appeared  on  a  TRACON  RDT  and 
the  time  the  expected  alert  appeared  on  tower 
RDT  #7.  This  problem  was  associated  with  the 
GSD/RDT  daisy  chain  configuration.  Again, 
Denver  AT  and  AF  personnel  are  aware  of  this 
problem. 

ACT-320  conducted  this  test  during  the  Build 
5B  OT&E  in  Orlando  (TAIO).  Again,  ACT-320 
used  TDWR  and  LLWAS  weather  scenario  test 
tapes,  developed  by  AOS-250  and  ACT-320, 
respectively,  to  stress  the  TDWR  system. 
ACT-320  played  the  weather  scenario  test 
tape,  and  monitored  the  TRACON  and  tower  GSDs 
and  RDTs  for  display  and  alarm/alert 
accuracy.  All  10  lines  of  each  RDT  were  used 
to  display  products  to  further  stress  the 
TDWR  system.  The  data  collected  during  the 
test  were  compared  against  expected  results 
and  proved  to  be  accurate. 
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OT&E  Subelement:  Human  Factors 

Test  Objective:  To  evaluate  the  interaction  of  air  traffic 

controllers  and  supervisors  with  the  TDWR  in 
the  operational  environment. 

Test  Participants:  ACT-320;  ASO-452;  Oklahoma  City,  Memphis,  and 

Houston  Air  Traffic;  NSSL 

Data  Collection  and 

Analysis  Method:  Statistical  analysis  was  performed  on  Air 

Traffic  Controller  responses  to 
questionnaire . 

Test  Description/Results : 

During  the  initial  OT&E  in  Oklahoma  City 
(TAl),  Oklahoma  City  air  traffic  controllers 
favorably  evaluated  the  GSD  and  RDT  with 
respect  to  the  displays  and  their  layout  of 
the  products.  However,  these  controllers  did 
not  operationally  use  the  TDWR  and  its 
products.  Therefore,  a  formal  ATC  evaluation 
was  conducted  in  Memphis  {TA4). 

The  AT  Evaluation  component  of  the  TDWR  OT&E 
Operational  test  was  accomplished  by  using  a 
questionnaire.  The  primary  purpose  of  this 
evaluation  was  to  determine  the  operational 
suitability  and  effectiveness  of  the  TDWR  in 
detecting  and  displaying  hazardous  weather 
products  in  and  around  the  airport  area.  A 
secondary  purpose  was  to  evaluate  the 
effectiveness  of  TDWR  in  the  management  of 
air  traffic  in  the  terminal  area  through  the 
forecast  and  display  of  gust  front  induced 
wind  shifts. 

Fifty-two  controllers,  supervisors,  and  staff 
specialists  (41  controllers,  8  supervisors 
and  3  staff  specialists)  were  briefed  on  the 
contents  of  the  TDWR  Questionnaire  prior  to 
the  start  of  the  21 -day  TDWR  AT  Evaluation. 
Two  sections  of  the  questionnaire  were 
designed  to  determine  the  operational 
suitability  and  effectiveness  of  both  the  GSD 
and  the  RDT;  each  section  contained  numerical 
value  questions  and  yes/no  questions. 

Comments  were  encouraged.  The  third  section 
consisted  of  general  open-ended  questions 
with  respect  to  the  overall  TDWR  system. 
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Build  4+,  a  "patch"  to  existing  TDWR  software 
that  displays  the  LLWAS  Center  Field  Wind 
(CFW)  data  on  the  RDT,  was  installed  and 
utilized  for  the  ATC  Evaluation. 

Of  the  AT  personnel  who  received  the 
questionnaire  briefing,  28  (54%)  returned 
completed  questionnaires.  Overall,  there  was 
a  favorable  response  to  the  TDWR.  Results 
and  comments  are  presented  by  evaluation 
category  below. 


Favorable 

Neutral 

Undecided 

Unfavorable 

GSD 

76% 

15% 

5% 

4% 

RDT 

77% 

8% 

7% 

8% 

System 

89.5% 

- 

3.5% 

7% 

Additional  response  analysis  is  included  in 
the  Quick  Look  Report  dated  September  1993. 

With  the  introduction  of  Build  5A,  another 
ATC  Evaluation  was  necessary  to  evaluate  the 
added  functionalities  of  Build  5A  and  their 
impact  on  the  AT  community.  This  evaluation 
was  conducted  in  Memphis  (TA6). 

Air  Traffic  Control  Specialists  (ATCS)  and 
supervisors  were  trained  on  the  Build  5A 
enhancement  prior  to  the  ATC  Evaluation. 
Again,  a  questionnaire  was  used  to  evaluate 
the  effectiveness,  suitability,  and 
useability  of  the  Build  5A  enhancements  from 
the  perspective  of  the  ATCSs  and  supervisors. 
The  questionnaire  was  divided  into  three 
sections  to  evaluate  the  Build  5A  enhancement 
with  respect  to  the  RDT,  the  GSD,  and  the 
enhancement  in  general. 

ACT-320  was  available  to  answer  user 
questions,  and  demonstrated  Build  5A 
enhancements.  In  addition,  ACT-320  monitored 
and  observed  tower  and  TRACON  ATCSs  and 
supervisors  while  using  the  integrated  TDWR — 
LLWAS  data. 

Twenty  evaluators  completed  and  returned  the 
questionnaire  during  the  7 -day  ATC 
Evaluation.  Overall,  there  was  a  favorable 
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response  to  the  Build  5 A  enhancement. 
Results  are  presented  by  evaluation  section 
below. 


Favorable 

Neutral 

Unfavorable 

RDT 

53% 

27% 

20% 

GSD 

66% 

29% 

5% 

General 

37% 

37% 

26% 

Additional  response  analysis  is  included  in 
the  Quick  Look  Report  dated  June  1994. 

Memphis  air  traffic  controllers  expressed  a 
concern  with  referring  to  LLWAS  winds  as 
"threshold  winds"  per  TDWR  documentation, 
when,  in  fact,  the  wind  sensors  are  located 
from  4,000  to  4,800  feet  from  the  runway. 
"Boundary  winds"  was  a  preferable  term  to 
refer  to  this  wind  information. 

Finally,  ACT-320  had  planned  to  conduct  a 
Build  5B  OT&E  Human  Factors  Evaluation  but 
was  unable  to  because  of  scheduling  conflicts 
in  Orlando  (TAIO). 
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OT&E  Subelement:  Safety 

Test  Objective:  To  estimate  or  determine  the  degree  to  which 

the  TDWR  meets  personnel  safety  requirements. 

Test  Description: 

Test  Participants:  ACT-320,  ACT-330,  AOS-250,  AMH-400 

Data  Collection  and 

Analysis  Method:  Gather  information  related  to  personnel 

safety  by  means  of  walk-throughs, 
inspections,  and  personnel  experiences. 

Test  Description/Results: 

The  Labor  Relations  and  Occupational  Safety 
Division,  AMH-400,  conducted  a  safety  walk¬ 
through  of  the  Oklahoma  City  facility  (TAl) 
and  revealed  potential  deficiencies  in 
pedestal-related  fall  hazards.  One  such 
hazard  was  the  absence  of  pedestal  ladder 
safety  climbing  devices.  AOS-250  and  AND-420 
have  initiated  an  action  plan  to  correct 
pedestal-related  safety  issues,  with 
particular  emphasis  on  pedestal  ladder  safety 
and  meeting  Occupational  Safety  and  Health 
Administration  (OSHA)  standards. 

The  Communications  and  Infrastructure  Branch, 
ACT-330,  performed  Lightning  Protection  and 
Grounding  Surveys  in  Houston  (TA4)  and 
Oklahoma  City  (TA2)  to  verify  compliance  of 
these  facilities  to  FAA  standards.  ACT-330 
identified  several  grounding  system  problems 
and  lightning  protection  deficiencies  which 
have  since  been  resolved. 
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OT&E  Subelement:  Maintainability 

Test  Objective:  To  determine  that  the  TDWR  system  is 

maintainable  utilizing  TIBs. 

Test  Participants:  ACT-320;  AOS-250;  ASO-452;  Eastern,  Southern, 

and  Southwest  Regions 

Data  Collection  and 

Analysis  Method:  To  recommend  changes  to  TIBs;  input  site 

latitude/longitude  into  solar  measurement 
routine . 

Test  Description/Results: 

Test  team  members  Initially  reviewed  the  TDWR 
TIBs  in  Oklahoma  City  (TAl).  The  alignment 
procedures  and  removal /replacement  procedures 
of  each  TIB  (System,  Transmitter,  Antenna, 
etc.)  were  performed  and  the  flow  paths  of 
the  Software  Users  Manual  for  the  RMS 
Software  were  verified  at  the  MDT.  Procedure 
deviations  and  recommended  changes  were 
prepared  as  SRs. 

Test  team  members  reviewed  updated  versions 
of  the  TIBs  for  procedure  accuracy  in  Houston 
(TA3)  and  provided  suggested  procedural 
changes.  Thereafter,  AOS-250  took  the 
responsibility  for  verifying  later  versions 
of  the  TIBs. 

The  solar  measurement  test  (sun  track), 
included  in  the  TDWR  System  TIB,  was  used  to 
determine  TDWR  pointing  accuracy.  ACT-320 
and  AOS-250  conducted  solar  measurement  tests 
first  in  Houston  (TA3)  and  later  in  Memphis 
(TA5).  Results  of  the  Houston  test  indicated 
that  both  the  procedure  and  the  software 
routine  were  incorrect.  After  subsequent 
procedure  and  routine  rework,  ACT- 320  and 
AOS-250  were  able  to  successfully  conduct  the 
solar  measurement  test  in  Memphis. 
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OT&E  Subelement:  Site  Adaptation  Data 

Test  Objective:  To  ensure  that  site  data  unique  to  each  site 

has  been  correctly  developed,  updated,  and 
installed  in  the  system. 

Test  Participants:  ACT-320,  AOS-250 

Data  Collection  and 

Analysis  Method:  MDT  screens  were  collected  to  be  compared 

against  site  survey  reports  when  available. 

Test  Description/Results : 

This  test  was  conducted  in  Oklahoma  City 
(TAl),  Memphis  (TA6),  Denver  (TA7  and  TA9), 
Kansas  City  (TAB),  and  Orlando  (TAIO).  ACT- 
320  collected  TDWR  MDT  Performance  and 
Computational  Parameters  Screens,  as  well  as 
antenna  stow  position  and  site  latitude  and 
longitude,  to  ensure  unique  site  data. 

Specifically,  Antenna,  Channel  A  and  B,  Data 
Processor,  and  DFU  Performance  Screens  and 
the  Computational  Parameters  Screen  were 
collected  to  determine  the  software  version 
number  of  the  various  TDWR  components.  These 
data  will  be  compared  against  site  survey 
reports  when  they  become  available. 
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OT&E  Subelement:  Security 

Test  Objective:  To  develop  the  baseline  security  model  for 

completing  the  FAA  required  SAC  for  TDWR,  to 
evaluate  and/or  develop  standard  practices 
and  procedures  for  required  security  controls 
and  countermeasures,  and  to  ensure  compliance 
with  DRR  issues . 

Test  Participants:  ACT-320,  AOS-250,  ASO-700,  AND-420,  ASO-452, 

Memphis  Airway  Facilities 

Data  Collection  and 

Analysis  Method:  Data  were  gathered  from  system  documentation, 

training  manuals,  performance  analyses,  and 
system  experts  who  represented  a  cross- 
section  of  system  technicians, 
administrators,  program  managers,  and 
security  specialists. 

Test  Description/Results: 

This  assessment  reviewed  potential  TDWR 
hazards  and  their  assessed  probability  of 
occurrence  and  severity  on  TDWR  operations 
and  air  traffic  operations  in  the  area. 

Also,  this  assessment  covered  very  general 
hazard  areas  that  could  affect  operations  and 
was  limited  only  to  security  hazards,  while 
excluding  hazards  that  had  already  undergone 
review  (e.g.  back-up  power,  uninterruptible 
power  supply,  air  conditioning,  lightning 
protection) . 

ASO-700  conducted  a  review  of  existing 
physical,  technical,  and  procedural  controls 
designed  to  mitigate  the  effects  of  the 
identified  hazards,  and  compared  the 
potential  hazards  with  identified 
countermeasures  to  determine  the  existence  of 
significant  security  vulnerabilities  that  may 
require  corrective  action  in  terms  of 
additional  technical,  physical,  and 
procedural  security  controls . 

Assessment  findings  were  extensive  and, 
therefore,  are  not  included  in  this  Final 
Report.  Assessment  findings  are  included  in 
the  TDWR  SAC  and  Accreditation  Documentation, 
prepared  by  the  Investigations  and  Internal 
Security  Branch  of  the  FAA  Civil  Aviation 
Security  Division,  ASO-700.  This 
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documentation  is  available  from  ACT-320  or 
ASO-700  (telephone  404-305-6801). 


B-21 


OT&E  Subelement: 


Transition  Switchover 


Test  Objective:  To  estimate  or  determine  that  the  TDWR  system 

and  procedures  are  such  that  a  move  from  the 
old  system  to  the  new  or  vice  versa  can  be 
accomplished  without  degrading  NAS  operations 
while  minimizing  the  impact  on  the  user. 

Test  Participants:  ACT-320;  ASO-452;  Memphis,  Denver,  Kansas 

City,  and  Orlando  Air  Traffic  and  Airvay 
Facilities 


Data  Collection  and 

Analysis  Method:  Monitored  MDT  and  GSDs  and  RDTs  for  system 

response . 

Test  Description/Results: 

The  transition  switchover  from  Build  4 
(baseline  stand-alone  TDWR  system)  to  Build 
4+  (which  displayed  LLWAS  centerfield  wind 
data  on  the  TDWR  RDT)  and  vice  versa  were 
tested  in  Memphis  (TA4).  Build  4+  consisted 
of  a  software  build  and  connectivity  between 
TDWR  and  LLWAS  through  a  DFU  serial  port. 

The  transition  switchover  was  successfully 
accomplished  with  minimal  impact  on  the  user. 

The  transition  switchover  from  Build  4+  to 
Build  5A  (which  interfaced  the  TDWR  with 
LLWAS  II)  and  vice  versa  was  also  tested  in 
Memphis  (TA6)  using  approved  test  procedures. 
Build  5A  consisted  of  a  software  build  and 
connectivity  between  the  TDWR  RPG  and  LLWAS 
and  connectivity  between  the  TDWR  DFU  and 
LLWAS.  The  transition  switchover  was 
successfully  accomplished  with  minimal  impact 
on  the  user. 

The  transition  switchover  from  stand-alone 
TDWR  (using  Build  5B)  and  LLWAS  systems  to  an 
integrated  TDWR — LLWAS  system  and  vice  versa 
was  tested  in  Denver  (TA7)  using  approved 
test  procedures.  The  difference  between  the 
integrated  TDWR — LLWAS  system  and  the  stand¬ 
alone  systems  included  a  software  build  and 
connectivity  between  the  TDWR  RPG  and  LLWAS 
and  connectivity  between  the  TDWR  DFU  and 
LLWAS.  The  transition  switchover  was 
successfully  accomplished  with  minimal  impact 
on  the  user. 
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No  transition  switchover  tests  were  needed 
for  Oklahoma  City  (TAl  and  TA2)/  Kansas  City 
(TA8),  and  Orlando  (TAIO)  because  OT&E  tests 
were  conducted  on  these  systems  immediately 
after  installation  and,  therefore,  there  was 
no  old  system  from  which  to  transition. 
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OT&E  Subelement:  Weather  Performance 


Test  Objective:  An  Investigative  Panel  (IP)/  a  group  of 

expert  radar  meteorologists  from  the  National 
Center  for  Atmospheric  Research  (NCAR), 
MIT/Lincoln  Laboratory,  and  NSSL  was  to 
verify  that  the  TDWR  met  operational 
suitability  and  reliability  requirements  in 
relation  to  its  ability  to  detect  hazardous 
weather;  to  assess  the  quality  of  the  base 
data;  to  verify  that  appropriate  alarms  were 
disseminated  according  to  system  design;  to 
determine  the  performance  of  the  automated 
weather  detection  algorithms;  and  to  evaluate 
a  set  of  NAS-SS-1000  requirements. 


Test  Participants;  ACT-320/  AOS-250/  NSSL,  NCAR,  MIT/Lincoln 

Laboratory 


Data  Collection  and 

Analysis  Method:  Record  base  data  and  archive  weather 

products;  determine  true  events  by  analyzing 
base  data;  compare  archived  weather  products 
to  true  events;  determine  hits,  misses,  and 
false  alarms;  perform  statistical  analysis  on 
algorithm  performance. 

Test  Description/Results : 

The  TDWR  Weather  Performance  was  originally 
evaluated  in  Oklahoma  City  (1992,  TAl). 
Although  limited  data  were  collected  for 
algorithm  performance  evaluation,  several 
goals  were  accomplished:  the  base  velocity 
and  reflectivity  data  were  of  good  quality, 
RDT  alarms  were  timely  and  accurately 
represented  algorithm  detections,  and  the  set 
of  NAS-SS-1000  requirements  were  verified. 

Several  issues  were  identified  during  1992 
Oklahoma  City  testing:  excess  clutter  residue 
breakthrough,  difficulty  in  clutter  residue 
editing  map  (CREM)  generation,  attenuation 
flagging,  excessive  gust  front  false  alarms 
and  misses,  excessive  microburst  false 
alarms,  and  questionable  microburst  shape 
parameters . 

The  lack  of  data  collected  in  Oklahoma  City 
(1992)  necessitated  additional  testing  during 
the  spring  of  1993  in  Oklahoma  City  (TA2). 

To  ensure  valid  analyses  of  the  Microburst 
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Detection  Algorithm  (MDA)  and  GFDA,  goals  for 
data  collection  and  algorithm  analysis  were 
set  at  300  microburst  and  100  gust  front 
events.  These  expectations  were  greatly 
exceeded;  880  microburst  and  256  gust  front 
events  were  collected. 

Algorithm  evaluation  included  determination 
of  hits,  misses,  and  false  alarms,  all  based 
on  comparison  of  ground  truth  determined  by 
meteorologists  and  detections  produced  by  the 
radar  algorithms.  Performance  statistics 
were  given  in  terms  of  Probability  of 
Detection  (POD)  and  False  Alarm  Ratio  (FAR) 
for  both  the  MDA  and  GFDA.  The  MDA  had  a  POD 
of  93%  (system  requirement  =  90%)  and  FAR  of 
18%  (design  goal  =  10%).  There  were  a  number 
of  detections  by  the  MDA  whose  validity  could 
not  be  ascertained  with  certainty.  These 
detections  were  often  in  areas  of  weak 
divergence  that  may  have  been  magnified  by 
spurious  data  points.  In  addition,  there 
were  a  significant  number  of  false  alarms  due 
to  range  folding.  The  IP  recommended 
proposed  remedies  for  microburst  false 
alarms,  the  implementation  of  which  had  the 
potential  to  reduce  the  FAR  to  an  acceptable 
value.  These  recommendations  were  tested  as 
part  of  AOS-250  Build  5B  OT&E  Shakedown 
testing.  AOS-250  test  results  are  included 
in  the  AOS-250  TDWR  Build  5B  OT&E  Shakedown 
Quick  Look  Report. 

Scored  on  an  event-by-event  basis,  the  GFDA 
had  a  POD  of  66%  and  a  FAR  of  24%.  While 
there  is  no  system  requirement  on  gust 
fronts,  these  figures  are  unacceptable.  The 
predominant  reasons  for  misses  and  false 
detections  were  range  folding  and  deallasing 
errors,  respectively.  While  there  are  no 
short-term  solutions,  continued  long-term 
research  may  result  in  more  robust  range  de¬ 
obscuration  editing  techniques  and  better 
dealiasing  techniques,  including  various 
multiple-pulse  repetition  frequency 
approaches.  Also,  there  were  several  false 
detections  in  the  presence  of  vertical  wind 
shear.  A  possible  remedy  would  be  to  add  a 
technique  to  the  algorithm  to  recognize  and 
remove  the  false  alarms  based  on  the 
knowledge  that  vertical  wind  shear  exists. 
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APPENDIX  C 


TDWR  TEST  VERIFICATION 
REQUIREMENTS  TRACEABILITY  MATRIX  (TVRTM) 


Table  C-1.  TDWR  Test  Verification  Requirements  Traceability  Matrix  (TVRTM) 


C-1 


-  To  be  accomplished  as  part  of  Weather  Processing  Test 


Table  C-1.  TDWR  Test  Verification  Requirements  Traceability  Matrix  (TVRTM) 


C-2 


-  To  be  accomplished  as  part  of  Weather  Processing  Test 


Table  C-1.  TDWR  Test  Verification  Requirements  Traceability  Matrix  (TVRTM) 
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-  To  be  accomplished  as  part  of  Weather  Processing  Test 


Table  C-1.  TDWR  Test  Verification  Requirements  Traceability  Matrix  (TVRTM) 


-  To  be  accomplished  as  part  of  Weather  Processing  Test 


Table  C-1.  TDWR  Test  Verification  Requirements  Traceability  Matrix  (TVRTM) 
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-  To  be  accomplished  as  part  of  Weather  Processing  Test 


Table  C-1.  TDWR  Test  Verification  Requirements  Traceabiiity  Matrix  (TVRTM) 
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-  To  be  accomplished  as  part  of  Weather  Processing  Test 


Table  C-1.  TDWR  Test  Verification  Requirements  Traceability  Matrix  (TVRTM) 
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-  To  be  accomplished  as  part  of  Weather  Processing  Test 


Table  C-1.  TDWR  Test  Verification  Requirements  Traceability  Matrix  (TVRTM) 
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-  To  be  accomplished  as  part  of  Weather  Processing  Test 


Table  C-1.  TDWR  Test  Verification  Requirements  Traceability  Matrix  (TVRTM) 
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-  To  be  accomplished  as  part  of  Weather  Processing  Test 


Table  C-1.  TDWR  Test  Verification  Requirements  Traceability  Matrix  (TVRTM) 
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-  Deferred  to  ACT-330 

-  To  be  accomplished  as  part  of  Weather  Processing  Test 


Table  C-1.  TDWR  Test  Verification  Requirements  Traceability  Matrix  (TVRTM) 
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-  Deferred  to  ACT-330 

-  To  be  accomplished  as  part  of  Weather  Processing  Test 


Table  C-1.  TDWR  Test  Verification  Requirements  Traceability  Matrix  (TVRTM) 
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-  Deferred  to  ACT-330 

-  To  be  accomplished  as  part  of  Weather  Processing  Test 


Table  C-1.  TDWR  Test  Verification  Requirements  Traceabrrty  Matrix  (TVRTM) 
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-  Deferred  to  ACT-330 

-  To  be  accomplished  as  part  of  Weather  Processing  Test 


Table  C-1.  TDWR  Test  Verification  Requirements  Traceability  Matrix  (TVRTM) 
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-  Deferred  to  ACT-330 

-  To  be  accomplished  as  part  of  Weather  Processing  Test 


Table  C-1.  TDWR  Test  Verification  Requirements  Traceability  Matrix  (TVRTM) 
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-  Deferred  to  ACT-330 

-  To  be  accomplished  as  part  of  Weather  Processing  Test 


Table  C-1.  TDWR  Test  Verification  Requitements  Traceability  Matrix  (TVRTM) 
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-  Deferred  to  ACT-330 

-  To  be  accomplished  as  part  of  Weather  Processing  Test 


Table  C-1.  TDWR  Test  Verification  Requirements  Traceabinty  Matrix  (TVRTM) 
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-  Deferred  to  ACT-330 

-  To  be  accomplished  as  part  of  Weather  Processing  Test 


Table  C-1 .  TDWR  Test  Verification  Requirements  T raceability  Matrix  (TVRTM) 


C-1 8 


-  Deferred  to  ACT-330 

-  To  be  accomplished  as  part  of  Weather  Processing  Test 


